• Experimental Design
• Power analyses
• Multi-factor ANOVA
• Nested ANOVA
• Factorial ANOVA
• Analysis of CoVariance (ANCOVA)

• In an experimental study the researcher assigns treatments to units
• In an observational study nature does the assigning of treatments to units
• The crucial advantage of experiments derives from the random assignment of treatments to units
• Random assignment, or randomization, minimizes the influence of confounding variables

• One possibility is that supplemental oxygen (explanatory variable) really does cause higher survival (response variable).
• The other is that the two variables are associated because other variables affect both supplemental oxygen and survival.
• Use of supplemental oxygen might be a benign indicator of a greater overall preparedness of the climbers that use it.
• Variables (like preparedness) that distort the causal relationship between the measured variables of interest (oxygen use and survival) are called confounding variables
• They are correlated with the variable of interest, and therefore preventing a decision about cause and effect.
• With random assignment, no confounding variables will be associated with treatment except by chance.

• The gold standard of experimental designs is the clinical trial
• Experimental design in all areas of biology have been informed by procedures used in clinical trials
• A clinical trial is an experimental study in which two or more treatments are assigned to human subjects
• The design of clinical trials has been refined because the cost of making a mistake with human subjects is so high
• Experiments on nonhuman subjects are simply called “laboratory experiments”or “field experiments”

• The researcher should randomize assignment of treatments to units or subjects
• Chance rather than conscious or unconscious decision determines which units end up receiving the treatment and which the control
• A completely randomized design is one in which treatments are assigned to all units by randomization
• Randomization breaks the association between possible confounding variables and the explanatory variable
• Randomization doesn’t eliminate the variation contributed by confounding variables, only their correlation with treatment
• Randomization ensures that variation from confounding variables is similar between the different treatment groups.

• Randomization should be carried out using a random process:
  • List all n subjects, one per row, in a computer spreadsheet.
  • Use the computer to give each individual a random number.
  • Assign treatment A to those subjects receiving the lowest numbers and treatment B to those with the highest numbers.
• Other ways of assigning treatments to subjects are almost always inferior because they do not eliminate the effects of confounding variables.
• “Haphazard” assignment, in which the researcher chooses a treatment while trying to make it random, has repeatedly been shown to be non-random and prone to bias.

• Blinding is the process of concealing information from participants (sometimes including researchers) about which subjects receive which treatment.
• Blinding prevents subjects and researchers from changing their behavior, consciously or unconsciously, as a result of knowing which treatment they were receiving or administering.
• For example, studies showing that acupuncture has a significant effect on back pain are limited to those without blinding (Ernst and White 1998).

• Reviews of medical studies have revealed that studies carried out without double- blinding exaggerated treatment effects by 16% on average compared with studies carried out with double-blinding (Jüni et al. 2001).
• Experiments on non–human subjects are also prone to bias from lack of blinding.
• Bebarta et al.(2003) reviewed 290 two-treatment experiments carried out on animals or on cell lines. The odds of detecting a positive effect of treatment were more than threefold higher in studies without blinding than in studies with blinding.
• Blinding can be incorporated into experiments on nonhuman subjects using coded tags that identify the subject to a “blind” observer without revealing the treatment (and who measures units from different treatments in random order).

• The goal of experiments is to estimate and test treatment effects against the background of variation between individuals (“noise”) caused by other variables
• One way to reduce noise is to make the experimental conditions constant
• In field experiments, however, highly constant experimental conditions might not be feasible nor desirable
• By limiting the conditions of an experiment, we also limit the generality of the results
• Another way to make treatment effects stand out is to include extreme treatments and to replicate the data.

• Blocking is the grouping of experimental units that have similar properties. Within each block, treatments are randomly assigned to experimental units.
• Blocking essentially repeats the same, completely randomized experiment multiple times, once for each block.
• Differences between treatments are only evaluated within blocks, and in this way the component of variation arising from differences between blocks is discarded.

• For example, consider the design choices for a two-treatment experiment to investigate the effect of clear cutting on salamander density.
• In the completely randomized (“two-sample”) design we take a random sample of forest plots from the population and then randomly assign each plot to either the clear-cut treatment or the no clear-cut treatment.
• In the paired design we take a random sample of forest plots and clear-cut a randomly chosen half of each plot, leaving the other half untouched.

• In the paired design, measurements on adjacent plot-halves are not independent. This is because they are likely to be similar in soil, water, sunlight, and other conditions that affect the number of salamanders.
• As a result, we must analyze paired data differently than when every plot is independent of all the others, as in the case of the two-sample design.
• Paired design is usually more powerful than completely randomized design because it controls for a lot of the extraneous variation between plots or sampling units that sometimes obscures the effects we are looking for.

• RCB design is analogous to the paired design, but may have more than two treatments. Each treatment is applied once to every block.
• As in the paired design, treatment effects in a randomized block design are measured by differences between treatments exclusively within blocks.
• By accounting for some sources of sampling variation blocking can make differences between treatments stand out.
• Blocking is worthwhile if units within blocks are relatively homogeneous, apart from treatment effects, and units belonging to different blocks vary because of environmental or other differences.

• Experimental studies are not always feasible, in which case we must fall back upon observational studies.
• The best observational studies incorporate as many of the features of good experimental design as possible to minimize bias (e.g., blinding) and the impact of sampling error (e.g., replication, balance, blocking, and even extreme treatments) except for one: randomization.
• Randomization is out of the question, because in an observational study the researcher does not assign treatments to subjects. Instead, the subjects come as they are.
• Two strategies are used to limit the effects of confounding variables on a difference between treatments in a controlled observational study: matching; and adjusting for known confounding variables (covariates).

• Type 1 error - \(\alpha\) - incorrectly rejecting a true null hypothesis
  • This is saying that there is an effect when there isn’t)
• Type 2 error - \(\beta\) - incorrectly accepting a false null hypothesis
  • This is saying that there isn’t an effect when there is)
• Power is the probability of rejecting a false null hypothesis
• Mostly we shoot for a power of around 80%
• Power can be calculated post hoc or a priori

\[ Power \propto \frac{(ES)(\alpha)(\sqrt n)}{\sigma}\]

• Power is proportional to the combination of these parameters

  • ES - effect size; how large is the change of interest?
  • alpha - significance level (usually 0.05)
  • n - sample size
  • sigma - standard deviation among experimental units within the same group.

• Read in the perchlorate data again
• Perform an ANOVA to test Strain on T4_Hormone_Level, but log-transform (base 10) the T4 variable

perc <- read.table('perchlorate_data.tsv', header=T, sep='\t')
x <- perc$Strain
y <- log10(perc$T4_Hormone_Level)

MyANOVA <- aov(y ~ x)
summary (MyANOVA)
boxplot(y ~ x)

• Consider the parameters of this test related to power:
  • The per-group sample sizes
  • The standard deviation (use the higher within-group sd)
  • The effect size (|difference between means| / within-grp sd)
• For more complex ANOVA power calculations (>2 groups):
  • The total variance
  • The within-group variance (use the higher one)

Based on your results, calculate the power for your ANOVA.

   pwr.t2n.test(n1=xxx, n2=xxx, d=xxx, sig.level=.05, power=NULL)

Check out the functions in the ‘pwr’ library (Unnecessary in this case, but could use ANOVA version):

   pwr.anova.test(k=2, n=190, f=0.25, sig.level=.05, power=NULL)

• effect size approximations:
  • f=0.1 (small)
  • f=0.25 (medium)
  • f=0.4 (large)
• see http://www.statmethods.net/stats/power.html

rnadata <- read.table('RNAseq.tsv', header=T, sep='')
head(rnadata)

• continuous response variable and two main effect categorical variables

gene <- rnadata$Gene80
microbiota <- rnadata$Microbiota
genotype <- rnadata$Genotype
boxplot(gene ~ microbiota)
boxplot(gene ~ genotype)
boxplot(gene ~ microbiota*genotype)

rna_aov <- aov(gene ~ microbiota + genotype + microbiota:genotype)
rna_aov <- aov(gene ~ microbiota*genotype)

• Examine the fitted model diagnostics and the ANOVA results table

plot(rna_aov)
summary(rna_aov)
anova(rna_aov)

• What are the general results of our hypothesis tests?
• If there is an interaction, can we understand it by looking at the boxplots?